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(54) Motion detection in an Interlaced video signal 



. (57)_„„ „ A motion decision value provides a dependable 
estimate whether motion occurs in a given region of a 
video image in an interlaced video sequence. The mo- 
tion detection is particularly applicable In the conversion 
from interlaced video to progressive video. An input first 
is fed to an absolute value former which computes a 
frame difference signal from a difference between the 
first field and the second field in one frame. A point-wise 
motion detection In the frame difference signal is then 



followed by a region-wise motion detection that c om- 
bines the point-wise motion detection signal with an ad- 
jacent point-wise motion detection signal delayed by 
one field. The motion decision value is then computed 
from the region-wise motion detection signal and output 
for further processing in the video signal processing sys- 
tem, such as for choosing whether the spatially interpo- 
lated video signal value or the temporally interpolated 
video signal value should be used for the output. 
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Description 

[0001] The present invention relates to a method of detecting motion in an interlaced video signal, the method com- 
prising receiving a series of video frames; and for each frame, determining a difference value for the difference between * 

s each of its pixels and corresponding pixels in the preceding frame, if any, and an apparatus therefor. 

[0002] Since different digital television (DTV) standards have been adopted in different parts of the world, it hs become 
desirable to employ video format conversion units in digital televisions. The ATSC DTV standard system of North 
America, for example, adopted 1080x1920 interlaced video, 720x1280 progressive video, 720x480 Interlaced and 
progressive video, and so on, as its standard video formats for digital TV broadcasting. 

10 [0003] Video format conversion refers to a signal processing operation in which an incoming video format Is converted 
to a different output video format so that the output video can be properly displayed on a displaying device such as a 
monitor, FLCD, or a plasma display, which has a fixed resolution. 

[0004] Video format conversion systems are of significant importance since the conversion can directly affect the 
visual quality of the video of a DTV receiver. Fundamentally, the video format conversion operation requires advanced 
is algorithms for multi-rate system design, poly-phase filter design and interiaced-to-progressive scanning rate conversion 
or simply deinteriacing. Deinteriacing is an operation that doubles the vertical scanning rate of the interlaced video 
signal. 

[0005] Interlaced video comprises a sequence of separately arriving fields, such as A1 , A2, A3, etc., where A1 and 
A3 are top imagea and A2 being a bottom image. The top and bottom images are aJso known as the odd and even 

20 fields. The most popular systems currently in use, namely NTSC, PAL, and SEC AM are two-field systems, where two 
consecutive fields (such as the top field A1 and the bottom field A2) make up a frame. Each scanned field contains, i. 
e., updates, every other line of a corresponding frame and the number of lines in the frame is twice the number of lines 
in each of the fields. Typically, the first field of a frame is identified with odd-numbered lines and the second field is 
identified with even-numbered lines. The fields are scanned onto the display screen one after the other at a defined 

25 frequency. 

[0006] By way of example, NTSC operates at 30 frames (60 fields of interlaced video) per second, with 525 lines 
per frame, and a horizontal to vertical aspect ratio of 4:3. The frame difference, therefore, is the difference between 
two fields having the same types (top or bottom) such as A1 and A3, or A2 and A4. PAL and SECAM operates at 25 
frames per second, with 625 lines per image, and the same aspect ratio of 4:3. As noted, the interlacing in all of these 

30 systems is 2:1, i.e., two fields per frame. The primary reason forthe interlacing of the lines between the fields is to 
reduce flicker in the display. An image that is updated, say, only 30 times a second would allow the human eye to 
perceive the scanning, because the image information would already start to fade before the next image is scanned 
onto the screen. When two fields are used, and each contains half of the information, the scanning rate In effect is 
raised to 60 Hz, and the human eye no longer perceives any flicker. 

35 [0007] Deinteriacing refers to the filling of unavailable lines in each of the fields A1 , A2, A3, and so on. As a result 
of deinteriacing, a 60 Hz field sequence (of interlaced video fields) becomes a 60 Hz progressive sequence. 
[0008] I nteriaced video is subject to several intrinsic drawbacks, referred to as artifacts. These include serrated lines 
that are observed when there is motion between fields, line flickering, raster line visibility, and field flickering. These 
also apply to DTV (digital TV) receivers. Historically, deinteriacing algorithms have been developed to enhance the 

40 video quality of NTSC TV receivers by reducing these intrinsic annoying artifacts of the interlaced video signal. Besides, 
elaborate deinteriacing algorithms utilizing motion detection or motion compensation provide excellent methods of 
doubling the vertical scanning rate of the interlaced video signal especially for stationary (motionless) objects in the 
video signal. 

[0009] The present invention therefore also relates to the motion detection based deinteriacing operation that can 

45 be used for analogue and digital TV receivers. 

[0010] The state of the art Includes a variety of deinteriacing algorithms, each having been exploited and studied 
comprehensively by many researchers during the last decade. Deinteriacing algorithms can be categorized into two 
classes, namely, 2-D (spatial) deinteriacing algorithms and 3-D (spatio-temporal) deinteriacing algorithms depending 
on the use of motion information embedded in a video sequence. Combined spatial and temporal 3-D deinteriacing 

50 algorithms based on a motion detection give more pleasing performance than 2-D deinteriacing algorithms. The key 
point of a 3-D de-interlacing algorithm is how to precisely detect motion in the interlaced video signals. The publications 
in the following list disclose some of the applicable de-intertacing methods. They may be categorized as follows: 

[1] Simple line doubling scheme, vertical filtering, vertical edge controlled interpolation method disclosed in the 
55 IEEE Transactions on Consumers Electronics, pp. 279-89, August 1 989 by D.I. Hentschei; 

[2] Edge direction dependent de-Interlacing method disclosed in the Proc. of the Int 'Workshop on HDTV, 1994, 
by D. Bagni, R Lancini, and S. Tubaro; 
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[3] Nonlinear interpolation methods based on: 

J a weighted median filter disclosed in the Proa of the IEEE ISCAS, pp. 433-36, Portland, USA, May 1 989, by 

J. Juhola, A. Nieminen, J. Sal, and Y. Neuvo, 
5 flR median hybrid Interpolation disclosed In Pro. Of SPIEs Visual Communications and Image Processing, 

Lausanne, Switzerland, October 1990, 00. 125-32 by A. Lehtonen and M. Renters, 

a complementary median filter disclosed in Proc. of the Int. Workshop on HDTV, 1994 by H. Blume, I. Schwo- 
erer t and K. Zygis, 

io [4] A motion adaptive method disclosed in IEEE Transactions on Consumer Electronics, pp. 110-114, May 1990 

by C. Markhauser. 

More recently, a new motion-detection-based de-interlacing method has been described in the following two 
patents: 

15 [5] US-A-5943099 

This patent discloses an interiaced-to-progressive conversion device which includes a spatial interpolator that 
provides for spatial interpolation and atemporal interpolator that provides for temporal interpolation of an interlaced 
video input signal. The system reduces jitter and related artifacts by temporally or spatially interpolating the signals. 



20 [6] US^A-595968r 

In this patent, two separate field memories are utilized for detecting rapid motion and slow motion in an inter- 
laced video sequence. An interlaced video signal is thereby converted Into a progressive-scanned signal. Differ- 
ences between spatial interpolation and temporal interpolation are used to determine whether the image is in 
motion, tf the differences exceed certain defined thresholds, motion is determined. The thresholds are dynamically 
25 adapted during the process. 

[001 1] The core of the methods described in the aforementioned two patents is to determine a motion decision factor 
based on the frame difference signal and the sample correlation in the vertical direction. These methods provide a way 

of reducing the visual artifacts that caiiarise from false motion detection by utilizing sample correlation In the vertical 

30 direction where the value is to be interpolated. A common drawback of these methods, however, is that they do not 
provide a true motion detection method when there are high frequency components in the vertical direction. In other 
words, when there are high frequency components in the vertical direction, the methods described in the aforemen- 
tioned patents will falsely determine that motion-containing pictures are being processed. 

[001 2] As a consequence, in many instances, these prior art processing methods do not provide for an increase in 
35 the vertical resolution even when no real motion is present between fields. 

[0013] A method according to the present invention is characterised by further processing said determined differ- 

ences, said further processing including deriving a motion signal for each pixel in dependence on the determined 

difference for that pixel and the determined difference for neighbouring pixels. 

[0014] Preferably, said further processing comprises low-pass filtering the determined difference value space com- 
40 prising said difference values for each frame and said motion signals are derived from the filtered difference values for 
each pixel and the filtered difference values for neighbouring pixels. 

[0015] Preferably, said further processing comprises comparing said filtered difference values with a reference to 
produce respective binary difference values and said motion signals are derived for each pixel from the result of ORing 
its binary difference value with those of said neighbouring pixels. 

45 [0016] Preferably,na method according to the present invention includes-tow-pass filtering the space comprising the 

results of said ORing for a frame to produce said motion signals. 

[0017] Said neighbouring pixels may be temporally or spatially neighbouring. 

[0018] The present Invention also provides an apparatus comprising means for performing each of the steps of a 
method according to the present invention. 
so [001 9] Embodiments of the present Invention, will now be described, by way of example, with reference to the ac- 
companying, drawings in which: 

Figure 1 is a diagrammatic view of two adjacent fields of a frame of an interlaced video signal; 
Figure 2 is a diagrammatic illustration of three fields for Illustrating the de-interlacing problem; 
55 Figure 3 is a more detailed view illustrating the de-interlacing process; 

Figure 4 is a block diagram illustrating the computation of a motion decision parameter; and 

Figure 5 is a block diagram showing the computation of the motion decision parameter and the resultant mixing 

of the spatially and temporally interpolated signals in dependence on the motion decision. 
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[0020] Referring to Figure 1 , an interlaced video signal is a series of frames each including a plurality of fields. As 
noted above, all conventional systems utilize two fields per frame which are sequentially scanned. A top field 1 contains 
Information regarding the first (v = 0), third (v = 2), fifth (v= 4), etc. lines, and a bottom field 2 contains information 
regarding the second (v= 1), fourth [v = 3), sixth (v= 5), etc. lines. 

5 [0021] In order to systematically describe the de-interlacing problem and the methods of the present Invention, let 
* x n denote the incoming interlaced video field at a time instant t = n and x n (v t h) denote the associated value of the 
video signal at the geometrical location ( v,/i). The variable vrepresents the vertical location and h represents horizontal 
location, in the cartesian co-ordinate system commonly used. By definition, the signal values of x n of the interlaced 
video signal are available only for the even lines (v = 0,2,4,...) if x n is the top field 1 . Similarly, the signal values of x n 

10 are available only for the odd lines of v (v= 1 ,3,5,...) if x n is the bottom field 2. Conversely, the signal values of x n are 
not available for odd lines if x n is a top field signal and the signal values of x n are not available for even lines if x n is a 
bottom field. Figure 1 shows the top field 1 scanned at t = m and the bottom field 2 scanned at t » m + 1 . Top and 
bottom fields are typically available in turn in the time axis, that is, pairs comprising top and bottom fields are used to 
make up frames. 

15 [0022] The de- interlacing function can be defined as a process to reconstruct or interpolate the non-available signal 
values of each field. That is : the de-interlacing function is to reconstruct the signal values of x n a\ the odd lines (v = 
1 ,3,5,...) for the top field x n and to reconstruct the signal values of x n atthe even lines (v = 0,2.4,...) forthe bottom field x^ 
[0023] Forthe simple description of the present invention, and to avoid any notational confusion in the disclosure, 
the de-interlacing problem will be simplified as a process which reconstructs or interpolates the unavailable signal 

20 value of x n at the I th line where the signal values of the lines at £1, i±3, i±5,... are available. More simply de-interlacing 
is to interpolate the value of xji t h), which was not originally available. It must be noted that, since x n . r and x n+f have 
a differentsampling phase from x n , the signal values of x^&h) and x^frh) are available, which is why motion detection 
can be incorporated with the de-interlacing function. This relationship is depicted in Figure 2, where dotted lines (and 
white circles) represent "no data available" and solid lines (and black circles) represent "available data.' 

25 [0024] Referring now to Figures 3, 4 and 5, there is illustrated a method of estimating a motion decision parameter 
m^h). Fundamentally, m^i,h) is estimated from the incoming interlaced video sequence and associated with the 
point-to-point degree of motion in the interlaced video sequence. The importance or the usefulness of estimating m n 
(i,h) can be easily understood from Figures 2 and 3. Suppose that precise motion detection information is available 
when we interpolate x^h) and suppose there is no motion at the spatial location (i,h) t then the best interpolation for 

30 x n (ijh) is to use the value of x^frh). This follows logically from the fact that no motion is introduced between f = n - 1 
and t =n + 1 at the spatial location (i,h), which very strongly implies that the value of x„(i t h) would be close to the value 
of x„_jftty. The usage of the motion decision parameter of the present invention is also to utilize the motion information 
for de-interlacing to property mix the temporal Information, which will be described later. 

[0025] First, the frame difference signal D n is computed by taking the difference between the fields in one frame 
35 interval as 

to which is associated with the scene change that occurred between the fields x tH , 1 and x^. The frame difference signal 
is then low-pass filtered to form 

45 

where LPF() represents a low-pass filtering process. The MxN kernel, in general, of the low-pass fitter LPF(). 
can be expressed as 
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where (iv 11( i^ N ) represents a set of predetermined coefficients. It should be noted that the characteristic of the 
low-pass fitter can be all-pass filter depending on the choice of the kernel W^. That is, if the kernel is set as M = N 
» 1 , and ^=1 , the LPF() becomes an all-pass filter and,- thus, d n - - - — 

[0026] Next, a point-wise motion detection signal is computed using 

f n {i,h)=T k (d n Uh)) (1) 

where denotes a threshold function represented by 



T(y)J Vdy * K 
* ' \0, otherwise 



2Q _. 



in which Kis a positive constant value. Then the region-wise motion detection signal is computed from the point-wise 
motion detection signals which logically combines the signals according to: 

_ M^)^ft)j^ 1 (M,/j)j|/^(/+1,/ J ) 



where denotes the one-field delayed motion detection signal described in (1) and where the notation II 

denotes the logical OR operation* 

[0027] Finally, the region-wise motion detection signal is low-pass filtered to form the motion decision parameter m n 
25 (i,h), namely: 



6 d 



30 



_ ..... I £60'+2xM + 2x<7).a M ( 2 ) 

where a,b,c,d £ 0, and a p q represents a set of normalized (i.e. 



), predetermined coefficients of a low-pass filter. For instance, the kernel of the low-pass filter used in (2) can be 
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[0028] Figure 4 illustrates the computation of the motion decision parameter mji,h) as described above. The com- 
50 puted motion decision parameter m^h) is then used to mix a spatially interpolated signal and a temporally interpolated 
signal, as illustrated In Figure 5. 

[0029] Referring to Figure 5, an Interpolation apparatus comprises a spatial interpolator 3, a temporal interpolator 4, 
a motion decision processor 5, and a mixer 6. The decision processor 5 corresponds to Figure 4 and includes, in the 
signal flow direction, an absolute value former 51 which defines the absolute difference parameter a first low-pass 
55 filter LPF 52 in which the filtering function W MxN with the MxN kernel is set; an adjustable or fixed threshold member 
53 which, in a preferred embodiment, is implemented by a controlled comparator; a buffer memory 54 and a further 
line memory 55 are con nected to an OR logic circuit 56 in which the function signal (Ji.h) is formed as described above; 
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finally, the motion detection signal m^h) is formed by low pass filtering in a spatial low-pass filter LPF 57. The output 
of the low-pass filter 57 is connected so that the motion detection signal mji.h) is supplied to a control input of the 
mixer 6. 

[0030] The spatial interpolator 3 spatially interpolates the value of xjf 9 h) by using a predetermined algorithm. The 
s temporal- interpolator 4 temporally interpolates the value of x n (l,h) by using a predetermined algorithm. The motion 
decision value m„Cf t h) computed in the motion decision processor 5, as described above, represents the degree of 
motion at the interpolation location {i,h). Conceptually, the value of the motion decision parameter will be bounded as 
0 ( m n(W 0 where the extreme m n (i,h) = 0 implies "no motion" and m n (i,h) = 1 implies "motion". The mixer mixes the 
output signal of the spatial interpolator and the output signal of the temporal interpolator in accordance with the motion 
*o decision value. Taking x* (i,h) and x* {i,h) the output signal of the spatial interpolator and the output signal of the temporal 
interpolator, respectively, the output signal of the mixer, or, the interpolated signal is represented by 
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x n (/,/j)=(1.m n (/,/7)).x^{/ / n) + m n (/,n).x^(/;h) 



[0031] It is clear that x^h)=x x (/,/?) when m^i t h)=0 (no motion) and x^i t h)=x s (i t h) when m n (/,h)=1 (motion). 
[0032] It will be understood thai it does not matter what kind of spatial interpolating algorithm (in the spatial interpolator 
3) and what kind of temporal interpolating algorithm (in the temporal interpolator 4) are used for the interpolation. The 
present invention is only concerned with estimating the motion decision value m n (i,h) and with mixing a spatially inter- 
so polated signal and a temporally interpolated signal in accordance with the estimated motion decision value. 

[0033] Specific information with regard to the interpolation of interlaced video signals and interlaced to progressive 
conversion is readily available to those of skill in the pertinent art. 
[0034] Some examples of the spatially Interpolated signal x s (/,n) are 



^iM. ,x ^ U) ^ ,,+1 ' h)) / 



which corresponds to a line average and 



x*(,;/>Wx„(/-l,h) 



which corresponds to a method known as line doubling. 
[0035] Also, some examples of temporally interpolated signal x 1 (i,h) 

n 

x n {i,n)= / 2 

and 
45 Claims 

1. A method of detecting motion in an interlaced video signal, the method comprising: 

receiving a series of video frames; and 
50 for each frame, determining a difference value for the difference between each of its pixels and corresponding I 

pixels in the preceding frame, tf any, ^ j 

characterised by further processing said determined differences, said further processing including deriving 
a motion signal for each pixel in dependence on the determined difference for that pixel and the determined dif- 
55 ference for neighbouring pixels. 

2. A method according claim 1 , wherein saidf urther processing comprises low-pass filtering the determined difference 
value space comprising said difference values for each frame and said motion signals are derived from the filtered 
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difference values for each pixel and the filtered difference values for neighbouring pixels. 

- 3. A method according to claim 2, wherein said further processing comprises comparing said filtered difference values 
with a reference to produce respective binary difference values and said motion signals are derived for each pixel 
s from the result of ORing Its binary difference value with those of said neighbouring pixels. 

4. A method according to claim 3, including low-pass filtering the space comprising the results of said ORing for a 
frame to produce said motion signals. 

10 5. A method according to arty preceding claim, wherein said neighbouring pixels are temporally neighbouring. 

6. A method according to any one of claims 1 to 4, wherein the neighbouring pixels are spatially neighbouring. 

7. In a video signal processing system, a method of computing a motion decision value, which comprises the following 
15 steps: 

inputting a video signal with an interlaced video sequence of fields; computing a frame difference signal from 
a difference between a previous field and a next field in the video sequence; 
forming a point-wise motion detection signal from the frame difference signal; 
20 computing a region-wise motion detection signal from the point-wise motion detection signal and an adjacent 

point-wise motion detection signal delayed by one field; and 

forming from the region-wise motion detection signal a motion decision value and outputting the motion deci- 
sion value for further processing in the video signal processing system. 

25 8. The method according to claim 7, which further comprises low-pass filtering the difference signal prior to the step 
of forming the point-wise motion detection signal. 

9. The method according to claim 8, wherein the step of low-pass filtering is defined by a low pass filter matrix 
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where *v n , ... represent a set of predetermined coefficients. 

10. The method according to claim 7, wherein the step of forming the point-wise motion detection signal comprises 
computing ... . .. 

/*<W=Wn(W) : 



where f n is the point-wise motion detection signal, / and h define a spatial location of the respective video signal 
value in a cartesian matrix, Trf-) denotes a threshold function represented as 



^ otherwise 



55 in which K is a positive constant, arid d^-) is the low-pass filtered frame difference signal. 

11. The method according to claim 7, wherein the region-wise motion detection signal is computed from the point- 
wise motion detection signal by logically combining the point-wise motion detection signal f„ as 
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*n(^)=/„(i«||/n-i('-1^>||/n.i(^1^) 

where /^(M.h) denotes the motion detection signal delayed by one field, the indices i and h define a spatial 
s location of the respective video signal value in a cartesian matrix, and the notation ii denotes a logical OR operation. 

12. The method according to claim 7, which further comprises low-pass filtering the region-wise motion detection signal 
prior to the outputting step. 

io 13. The method according to claim 1 2, wherein the region-wise motion detection signal is low-pass filtered to form the 
motion decision value m n (i,h) by: 

b d 

where a s b,c,d £ 0, and a pq represents a set of normalized predetermined coefficients of a low pass filter. 
20 14. The method according to claim 1 3, which comprises defining a kernel of the low pass filter as 
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15. In a method of processing interlaced video signals, which comprises: 



spatially interpolating a value of the video signal at a given location from a video signal of at least one adjacent 
location in a given video field; 

temporally interpolating the value of the video signal at the given location from a video signal at the same 
location in temporally adjacent video fields; and 
35 forming a motion decision value for the same location in accordance with claim 7; and 

mixing an output signal for the video signal at the given location from the spatially interpolated signal and the 
temporally interpolated signal and weighting the output signal in accordance with the motion decision value. 

16. The method according to claim 15, which comprises varying the motion decision value between 0 and 1 as a 
40 function of an estimate of the degree of motion at the given location and, upon estimating a high degree of motion, 

heavily weighting the output signal towards the spatially interpolated signal and, upon estimating a low degree of 
motion, heavily weighting the output signal towards the temporally interpolated signal. 

17. The method according to claim 1 6, which comprises outputting the spatially interpolated signal as the output signal 
45 upon estimating a high degree of motion, and outputting the temporally interpolated signal as the output signal 

upon estimating a low degree of motion. 

18. In a video signal processing system, an apparatus for computing a motion decision value, comprising: 

50 an input for receiving a video signal with an interlaced video sequence; 

difference forming means connected to said input for computing a frame difference signal from a difference 
between a previous field and a next field of a current field to be deinteriaced; 

means for forming a point-wise motion detection signal from the frame difference signal, and for computing a 
region-wise motion detection signal from the point-wise motion detection signal and an adjacent point-wise 
55 motion detection signal delayed by one field; and 

means forforming from the region-wise motion detection signal a motion decision value and for outputting the 
motion decision value for further processing in the video signal processing system. 
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19. The apparatus according to claim 1 8, which further comprises a low-pass fitter connected to said difference forming 
means. 

20. The apparatus according to claim 19, wherein said low-pass fitter is programmed with a low pass filter matrix 
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where iv 11f ... w MN represent a set of predetermined coefficients. 

21 . The apparatus according to claim 1 8, which comprises a logic member programmed to compute the motion decision 
valuef rom the point-wise motion detection signal by logically combiningthe point-wise motion detection signal / n as 



20 



4n( WbfnV>h)\\fn-i (« '*)||//m 



where /^i(M,ft) denotes the motion detection signal delayed by one field, the Indices / and h define a spatial 
location of the respective video signal value in a cartesian matrix, and the notation )i denotes a logical OR operation. 

25 22. The apparatus according to claim 18, which further comprises a low-pass filter connected to an output of said 
outputting means. 

23. The apparatus according to claim 22, wherein said low-pass filter is programmed to filter the region-wise motion 
detection signal to form the motion decision value m^h) by: 
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where a,b,c,d £ 0, and a A<7 represents a set of normalized predetermined coefficients of a low pass filter. 
24. The apparatus according to claim 23, wherein said low-pass filter is defined with a kernel 
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25. An apparatus of processing interlaced video signals, which comprises: 

an input for receiving a video signal with an interlaced video sequence of fields; 

a spatial interpolator connected to said input and configured to spatially interpolate a value of the video signal 
at a given location from a video signal of at least one adjacent location in a given video field; 
a temporal interpolator connected to said input in parallel with said spatial interpolator for temporally interpo- 
lating the value of the video signal at the given location from a video signal at the same location In temporally 
adjacent video fields; and 

a computing apparatus according to claim 12 connected to said Input and in parallel wtth said spatial interpo- 
lator and said temporal interpolator for forming a motion decision value for the same location; and 
a mixer connected to receive an output signal from each of said spatial interpolator, said temporal interpolator, 
and said computing apparatus, said mixer being configured to mix an output signal for the video signal at the 
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given location from the spatially interpolated signal and the temporally interpolated signal In dependence 
the motion decision value output by said computing apparatus. 
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